Chip 1998 September

home *** CD-ROM | disk | FTP | other *** search

/ Chip 1998 September / CHIP Eylül 1998.iso / Slackwar / docs / Virtual-Services-HOWTO < prev next >

Wrap

Text File | 1997-11-14 | 28.4 KB | 943 lines

Virtual Services Howto Brian Ackerman, brian@nycrc.net v1.2, 4 November 1997 This document came about to satisfy the ever increasing need to know how to virtualize a service. 1. Introduction 1.1. Knowledge Required Creating a virtual services machine is not all that difficult. However, more than fundamental knowledge is required. And this document is not a primer to how to fully configure a Linux machine. In order to understand this HOWTO document it is assumed that you are thoroughly familiar with the following: ╖ Compiling a Linux kernel and adding IP aliasing support IP alias mini-HOWTO ╖ Setting up and configuring of network devices NET-3 HOWTO ╖ Setting up of inetd NET-3 HOWTO ╖ Compiling and installing various network packages like Sendmail Site Apache Site Wu-Ftpd FAQ ╖ Setting up DNS DNS HOWTO If you are uncertain of how to proceed with any of the above it is STRONGLY recommended that you use the links provided to familiarize yourself with all packages. I will NOT reply to any mail reguarding any of the above. Please direct any questions to the appropriate author of the HOWTO. 1.2. Purpose The purpose of virtual services is to allow a single machine to recognize multiple IP addresses without multiple network cards. IP aliasing is a kernel option that allows you to assign each network device more than one IP address. The kernel then multiplexes (swaps between them very fast) in the background and to the user it appears like you have more than one network card. This multiplexing allows multiple domains (www.domain1.com, www.domain2.com, etc.) to be hosted by the same machine for the same cost as hosting one domain. Unfortunately, most services (ftp, web, mail) were not designed to handle muliple domains. In order to make them work properly you must modify both configuration files and source code. This document describes how to make these modifications in the setting up of a virtual machine. A deamon is also required in order to make virtual services function. The source for this daemon (virtuald) is provided later in this document. 1.3. Feedback This document will expand as packages are updated and source or configuration modifications change. If there are any portions of this document that are unclear please feel free to email me with your suggestions or questions. So that I do not have to go searching through the entire HOWTO please make certain that all comments are as specific as possible and include the section where the uncertainty lies. It is important that all mail be addressed with VIRTSERVICES HOWTO in the subject line. Any other mail will be considered personal and all my friends know that I do not ever read my personal mail so it will probably get discarded with theirs. Please note that my examples are just that, examples and should not be copied verbatim. You may have to insert your own values. If you are having trouble, send me mail, with all the pertinent configuration files and the error messages you get when installing, and I will look them over and mail my suggestions back. 1.4. Revision History V1.0 Initial version V1.1 Fixed error in Virtual Web Section V1.2 Fixed the date 1.5. Copyright/Distribution This document is Copyright (c) 1997 by The Computer Resource Center Inc. A verbatim copy may be reproduced or distributed in any medium physical or electronic without permission of the author. Translations are similiarly permitted without express permission if it includes a notice on who translated it. Commercial redistribution is allowed and encouraged; however please notify Computer Resource Center of any such distributions. Excerpts from the document may be used without prior consent provided that the derivative work contains the verbatim copy or a pointer to a verbatim copy. Permission is granted to make and distribute verbatim copies of this document provided the copyright notice and this permission notice are preserved on all copies. In short, we wish to promote dissemination of this information through as many channels as possible. However, I do wish to retain copyright on this HOWTO document, and would like to be notified of any plans to redistribute this HOWTO. 2. IP aliasing IP aliasing is a kernel option that needs to be set up in order to run a virtual hosting machine. There is already a mini-HOWTO on IP aliasing. Consult that for any questions on how to set it up. 3. Virtuald 3.1. How it works Every network connection is made up of two IP address/port pairs. The API (Applications Program Interface) for network programming is called the Sockets API. The socket acts like an open file and by reading/writing to it you can send data over a network connection. There is a function call getsockname that will return the IP address of the local socket. Virtuald uses getsockname to determine which IP on the local machine is being accessed. Virtuald reads a config file to retrieve the directory associated with that IP. It will chroot to that directory and hand the connection off to the service. Chroot resets / or the root directory to a new point so everything higher in the directory tree is cut off from the running program. Therefore, each IP address gets their own virtual filesystem. To the network program this is transparent and the program will behave like nothing happened. Virtuald in conjunction with a program like inetd can then be used to virtualize any service. 3.2. inetd Inetd is a network super server that listens at multiple ports and when it receives a connection (for example, an incoming pop request), inetd performs the network negotiation and hands the network connection off to the specified program. This prevents servers from running idly when they are not needed. A standard /etc/inetd.conf file looks like this: ftp stream tcp nowait root /usr/sbin/tcpd wu.ftpd -l -a pop-3 stream tcp nowait root /usr/sbin/tcpd in.qpop -s A virtual /etc/inetd.conf file looks like this: ftp stream tcp nowait root /usr/bin/virtuald virtuald /virtual/conf.ftp wu.ftpd -l -a pop-3 stream tcp nowait root /usr/bin/virtuald virtuald /virtual/conf.pop in.qpop -s 3.3. virtual.conf Each service gets a conf file that will control what IPs and directories are allowed for that service. You can have one master conf file or several conf files if you want each service to get a different list of domains. A virtual.conf file looks like this: # This is a comment and so are blank lines # Format IP <SPACE> dir <NOSPACES> 10.10.10.129 /virtual/foo.bar.com 10.10.10.130 /virtual/bar.foo.com 10.10.10.157 /virtual/boo.la.com 3.4. The source (virtuald) #include <netinet/in.h> #include <sys/socket.h> #include <arpa/inet.h> #include <stdarg.h> #include <string.h> #include <syslog.h> #include <stdio.h> #define BUFSIZE 8192 main(int argc,char **argv) { char buffer[BUFSIZE]; char *ipaddr,*dir; logit("Virtuald Starting: $Revision: 1.21 $"); if (!argv[1]) { logit("invalid arguments: no conf file"); quitting_virtuald(0); } if (!argv[2]) { logit("invalid arguments: no program to run"); quitting_virtuald(0); } if (getipaddr(&ipaddr)) { logit("getipaddr failed"); quitting_virtuald(0); } sprintf(buffer,"Incoming ip: %s",ipaddr); logit(buffer); if (iptodir(&dir,ipaddr,argv[1])) { logit("iptodir failed"); quitting_virtuald(0); } if (chroot(dir)<0) { logit("chroot failed: %m"); quitting_virtuald(0); } sprintf(buffer,"Chroot dir: %s",dir); logit(buffer); if (chdir("/")<0) { logit("chdir failed: %m"); quitting_virtuald(0); } if (execvp(argv[2],argv+2)<0) { logit("execvp failed: %m"); quitting_virtuald(0); } } int logit(char *buf) { openlog("virtuald",LOG_PID,LOG_DAEMON); syslog(LOG_ERR,buf); closelog(); return 0; } int quitting_virtuald(int retval) { exit(retval); return 0; } int getipaddr(char **ipaddr) { struct sockaddr_in virtual_addr; static char ipaddrbuf[BUFSIZE]; int virtual_len; char *ipptr; virtual_len=sizeof(virtual_addr); if (getsockname(0,(struct sockaddr *)&virtual_addr,&virtual_len)<0) { logit("getipaddr: getsockname failed: %m"); return -1; } if (!(ipptr=inet_ntoa(virtual_addr.sin_addr))) { logit("getipaddr: inet_ntoa failed: %m"); return -1; } strncpy(ipaddrbuf,ipptr,sizeof(ipaddrbuf)-1); *ipaddr=ipaddrbuf; return 0; } int iptodir(char **dir,char *ipaddr,char *filename) { char buffer[BUFSIZE],*bufptr; static char dirbuf[BUFSIZE]; FILE *fp; if (!(fp=fopen(filename,"r"))) { logit("iptodir: fopen failed: %m"); return -1; } *dir=NULL; while(fgets(buffer,BUFSIZE,fp)) { buffer[strlen(buffer)-1]=0; if (*buffer=='#' || *buffer==0) continue; if (!(bufptr=strchr(buffer,' '))) { logit("iptodir: strchr failed"); return -1; } *bufptr++=0; if (!strcmp(buffer,ipaddr)) { strncpy(dirbuf,bufptr,sizeof(dirbuf)-1); *dir=dirbuf; break; } } if (fclose(fp)==EOF) { logit("iptodir: fclose failed: %m"); return -1; } if (!*dir) { logit("iptodir: ip not found in conf file"); return -1; } return 0; } 4. Virt scripts 4.1. virtfs Each domain should get their own directory structure. Since you are using chroot you will require duplicate copies of the shared libraries, binaries, conf files, etc. I use /virtual/domain.com for each domain that I create. I realize that you are taking up more disk space but it is cheaper than a whole new machine and network cards. If you really want to preserve space you can link the files together so only one copy of each binary exists. Here is a sample virtfs script: #!/bin/bash echo '$Revision: 1.21 $' echo -n "Enter the domain name: " read domain if [ "$domain" = "" ] then echo Nothing entered: aborting exit 0 fi leadingdir=/virtual echo -n "Enter leading dir: (Enter for default: $leadingdir): " read ans if [ "$ans" != "" ] then leadingdir=$ans fi newdir=$leadingdir/$domain if [ -d "$newdir" ] then echo New directory: $newdir: ALREADY exists exit 0 else echo New directory: $newdir fi echo Create $newdir mkdir -p $newdir echo Create bin cp -pdR /bin $newdir echo Create dev cp -pdR /dev $newdir echo Create dev/log ln -f /virtual/log $newdir/dev/log echo Create etc mkdir -p $newdir/etc for i in /etc/* do if [ -d "$i" ] then continue fi cp -pd $i $newdir/etc done echo Create etc/skel mkdir -p $newdir/etc/skel echo Create home for i in a b c d e f g h i j k l m n o p q r s t u v w x y z do mkdir -p $newdir/home/$i done echo Create home/c/crc mkdir -p $newdir/home/c/crc chown crc.users $newdir/home/c/crc echo Create lib mkdir -p $newdir/lib for i in /lib/* do if [ -d "$i" ] then continue fi cp -pd $i $newdir/lib done echo Create proc mkdir -p $newdir/proc echo Create sbin cp -pdR /sbin $newdir echo Create tmp mkdir -p -m 0777 $newdir/tmp chmod +t $newdir/tmp echo Create usr mkdir -p $newdir/usr echo Create usr/bin cp -pdR /usr/bin $newdir/usr echo Create usr/lib mkdir -p $newdir/usr/lib echo Create usr/lib/locale cp -pdR /usr/lib/locale $newdir/usr/lib echo Create usr/lib/terminfo cp -pdR /usr/lib/terminfo $newdir/usr/lib echo Create usr/lib/zoneinfo cp -pdR /usr/lib/zoneinfo $newdir/usr/lib echo Create usr/lib/\*.so\* cp -pdR /usr/lib/*.so* $newdir/usr/lib echo Create usr/sbin cp -pdR /usr/sbin $newdir/usr echo Linking usr/tmp ln -s /tmp $newdir/usr/tmp echo Create var mkdir -p $newdir/var echo Create var/lock cp -pdR /var/lock $newdir/var echo Create var/log mkdir -p $newdir/var/log echo Create var/log/wtmp cp /dev/null $newdir/var/log/wtmp echo Create var/run cp -pdR /var/run $newdir/var echo Create var/run/utmp cp /dev/null $newdir/var/run/utmp echo Create var/spool cp -pdR /var/spool $newdir/var echo Linking var/tmp ln -s /tmp $newdir/var/tmp echo Create var/www/html mkdir -p $newdir/var/www/html chown webmast.www $newdir/var/www/html chmod g+s $newdir/var/www/html echo Create var/www/master mkdir -p $newdir/var/www/master chown webmast.www $newdir/var/www/master echo Create var/www/server mkdir -p $newdir/var/www/server chown webmast.www $newdir/var/www/server exit 0 4.2. virtexec To execute commands in a virtual environment you have to chroot to that directory and then run the command. I have written a special shell script called virtexec that handles this for any command: #!/bin/sh echo '$Revision: 1.21 $' BNAME=`basename $0` FIRST4CHAR=`echo $BNAME | cut -c1-4` REALBNAME=`echo $BNAME | cut -c5-` if [ "$BNAME" = "virtexec" ] then echo Cannot run virtexec directly: NEED a symlink exit 0 fi if [ "$FIRST4CHAR" != "virt" ] then echo Symlink not a virt function exit 0 fi list="" num=1 for i in /virtual/* do if [ ! -d "$i" ] then continue fi if [ "$i" = "/virtual/lost+found" ] then continue fi list="$list $i $num" num=`expr $num + 1` done if [ "$list" = "" ] then echo No virtual environments exist exit 0 fi dialog --clear --title 'Virtexec' --menu Pick 20 70 12 $list 2> /tmp/menu.$$ if [ "$?" = "0" ] then newdir=`cat /tmp/menu.$$` else newdir="" fi tput clear rm -f /tmp/menu.$$ echo '$Revision: 1.21 $' if [ ! -d "$newdir" ] then echo New directory: $newdir: NOT EXIST exit 0 else echo New directory: $newdir fi echo bname: $BNAME echo realbname: $REALBNAME if [ "$*" = "" ] then echo args: none else echo args: $* fi echo Changing to $newdir cd $newdir echo Running program $REALBNAME chroot $newdir $REALBNAME $* exit 0 Please note that you must have the dialog program installed on your system for this to work. To use virtexec just symlink a program to it. For example, ln -s /usr/bin/virtexec /usr/bin/virtpasswd ln -s /usr/bin/virtexec /usr/bin/virtvi ln -s /usr/bin/virtexec /usr/bin/virtpico ln -s /usr/bin/virtexec /usr/bin/virtemacs ln -s /usr/bin/virtexec /usr/bin/virtmailq Then if you type virtvi or virtpasswd or virtmailq it will allow you to vi a program, change a user's password or check the mail queue on your virtual system. You can create as many virtexec symlinks as you want. However, note that if your program requires a shared library it has to be in the virtual filesystem. The binary has to exist on the virtual filesystem also. 4.3. Notes on virtfs and virtexec I install all the scripts in /usr/bin. Anything that I do not want to put on the virtual filesystem I put in /usr/local. The script does not touch anything in there for copying. Any files that are important to not cross virtual filesystems should be removed. For example, ssh is installed on my system and I did not want the private key for the server available on all the virtual filesystems so I remove it from each virtual filesystem after I run virtfs. I also change resolv.conf and remove anything that has the name of another domain on it for legal reasons. For example, /etc/hosts and /etc/HOSTNAME. The programs that I symlink to virtexec are: ╖ virtpasswd -- change a user password ╖ virtadduser -- create a user ╖ virtdeluser -- delete a user ╖ virtsmbstatus -- see samba status ╖ virtvi -- edit a file ╖ virtmailq -- check out the mailq ╖ virtnewaliases -- rebuild alias tables 5. DNS You can configure DNS normally. The beauty of this system is that all services will behave normally like they are on separate machines. There is a HOWTO on DNS. 6. Syslog 6.1. Problem Syslog is the system logging utility commonly used on UNIX systems. Syslog is a daemon that opens a special file called a FIFO. A FIFO is a special file that is like a pipe. Anything that is written to the write side will come out the read side. The syslog daemon waits for data from the read side. There are C functions that write to the write side. If you write your program with these C functions your output will go to syslog. Remember that we have used a chroot environment and the FIFO /dev/log is not in the virtual environment. That means all the virtual environments will not log to syslog. We cannot simply copy the file since the programs use /dev/log instead of the new one we would create. Beware that certain versions of syslog use a udp socket instead of the FIFO. However, this is usually not the case. 6.2. Solution Syslog can look to a different FIFO if you tell it on the command line so run syslog with the argument: syslog -p /virtual/log Then link /dev/log to /virtual/log by (Note it is a SYMLINK): ln -sf /virtual/log /dev/log Then link all the /dev/log copies to this file by running (Note it is a hard link and NOT a symlink): ln /virtual/log /virtual/domain.com/dev/log The virtfs script above already does this. Since /virtual is one contiguous disk and the /dev/log's are linked they have the same inode number and point to the same data. The chroot cannot stop this so all your virtual /dev/log's will now function. Note that all the messages from all the environments will be logged in one place. However, you can write separate programs to filter out the data. If you do not want to write a program and require separate log files you can use a separate syslog for each virtual filesystem by running: syslog -p /virtual/domain1.com/dev/log syslog -p /virtual/domain2.com/dev/log However that wastes process id's so I do not recommend it. This version of the syslog.init file relinks the /dev/log's each time you start it in case they have been improperly set up. Here is a modified syslog.init file: #!/bin/sh # Source function library. \&. /etc/rc.d/init.d/functions case "$1" in start) echo -n "Starting dev log: " ln -sf /virtual/log /dev/log echo done echo -n "Starting system loggers: " daemon syslogd -p /virtual/log daemon klogd echo echo -n "Starting virtual dev log: " for i in /virtual/* do if [ ! -d "$i" ] then continue fi if [ "$i" = "/virtual/lost+found" ] then continue fi ln -f /virtual/log $i/dev/log echo -n "." done echo " done" touch /var/lock/subsys/syslog ;; stop) echo -n "Shutting down system loggers: " killproc syslogd killproc klogd echo rm -f /var/lock/subsys/syslog ;; *) echo "Usage: syslog {start|stop}" exit 1 esac exit 0 Note that you do not have to put all the virtual filesystems on one disk. However, you will have to run a different syslog for each partition that has virtual filesystems on it. 7. Virtual FTP Wu-ftpd comes with built in support to make it virtual. However, you cannot maintain separate password files for each domain. For example, if bob@domain1.com and bob@domain2.com both want an account you would have to make one of them bob2 or have one of the users choose a different user name. Since you now have a virtual filesystem for each domain you have separate password files and this problem goes away. Just create a virtnewuser script and virtpasswd script in the way mentioned above and you are all set. You can also have anonymous ftp in each virtual environment as that would be unaffected by the virtual filesystem as well. The inetd.conf entries for wu-ftp: ftp stream tcp nowait root /usr/bin/virtuald virtuald /virtual/conf.ftp wu.ftpd -l -a 8. Virtual Web Apache has their own support for virtual domains. This is the only program I recommend using the internal virtual domain mechanism. When you run something through inetd there is a cost. The program now has to start up each time you run it. That means slower response times which is unacceptable for web service. Apache also has a mechanism for stopping connections when too many come in. However, if you did want to run Apache through inetd then add the following line to your inetd.conf file: www stream tcp nowait www /usr/bin/virtuald virtuald /virtual/conf.www httpd -f /var/www/conf/httpd.conf In the /var/www/conf/httpd.conf file you have to specify: ServerType inetd Then configure each instance of the Apache server like you would normally for single domain use. At the time of this writing there is no virtual web HOWTO. However, I am under the impression one is coming. Eventually I will just refer to that HOWTO and have some notes on it. If it does not come soon and I get enough requests I will write a small section on how to configure the Apache virthost directive. 9. Virtual Mail/Pop 9.1. Qmail Notice This section applies to sendmail only. A section for qmail will be added in the next version of this HOWTO document. 9.2. Problem Virtual mail support is in ever increasing demand. Sendmail says it supports virtual mail. What it does support is listening for incoming mail from different domains. You can then specify to have the mail forwarded somewhere. However, if you forward it to the local machine and have incoming mail to bob@domain1.com and bob@domain2.com they will go to the same mail folder. This is a problem since both bob's are different people with different mail. 9.3. Bad Solution You can make sure that each user name is unique by using a numbering scheme: bob1, bob2, etc or prepending a few characters to each username dom1bob, dom2bob, etc. You could also hack mail and pop to do these conversions behind the scenes but that can get messy. Outgoing mail also has the banner maindomain.com and you want each subdomain's outgoing mail banner to be different. 9.4. Good Solution Each virtual filesystem gives a domain its own /etc/passwd. This means that bob@domain1.com and bob@domain2.com are different users in different /etc/passwds so mail will be no problem. They also have their own spool directories so the mail folders will be different files on different virtual filesystems. However, sendmail requires one minor source code modification. Sendmail has a file called /etc/sendmail.cw and it contains all machine names that sendmail will deliver mail to locally rather than forwarding to another machine. Sendmail does internal checking of all the devices on the machine to initialize this list with the local IPs. This presents a problem if you are mailing between virtual domains on the same machine. Sendmail will be fooled into thinking another virtual domain is a local address and spool the mail locally. For example, bob@domain1.com sends mail to fred@domain2.com. Since domain1.com's sendmail thinks domain2.com is local, it will spool the mail on domain1.com and never send it to domain2.com. You have to modify sendmail (I did this on v8.8.5 without a problem): vi v8.8.5/src/main.c # Approximately Line 494 It should say: load_if_names(); Replace it with: /* load_if_names(); Commented out since hurts virtual */ Note only do this if you need to send mail between virtual domains which I think is probable. This will fix the problem. However, the main ethernet device eth0 is not removed. Therefore, if you send mail from a virtual IP to the one on eth0 on the same box it will delivery locally. Therefore, I just use this as a dummy IP virtual1.domain.com (10.10.10.157). I never send mail to this host so neither will the virtual domains. This is also the IP I would use to ssh into the box to check if the system is ok. Edit /etc/sendmail.cw with the local hostnames. vi /etc/sendmail.cw mail.domain1.com domain1.com domain1 localhost Create /etc/sendmail.cf like you would normally through m4. I used: divert(0)dnl VERSIONID(`@(#)tcpproto.mc 8.5 (Berkeley) 3/23/96') OSTYPE(linux) FEATURE(redirect) FEATURE(always_add_domain) FEATURE(use_cw_file) FEATURE(local_procmail) MAILER(local) MAILER(smtp) Edit /etc/sendmail.cf to respond as your virtual domain: vi /etc/sendmail.cf # Approximately Line 86 It should say: #Dj$w.Foo.COM Replace it with: Djdomain1.com Sendmail cannot be started stand alone anymore so you have to run it through inetd. This is inefficient and will result in lower start up time but if you had such a high hit site you would not share it on a virtual box with other domains. Note that you are NOT running with the -bd flag. Also note that you need a sendmail -q running for each domain to queue up undelivered mail. The new sendmail.init file: #!/bin/sh # Source function library. \&. /etc/rc.d/init.d/functions case "$1" in start) echo -n "Starting sendmail: " daemon sendmail -q1h echo echo -n "Starting virtual sendmail: " for i in /virtual/* do if [ ! -d "$i" ] then continue fi if [ "$i" = "/virtual/lost+found" ] then continue fi chroot $i sendmail -q1h echo -n "." done echo " done" touch /var/lock/subsys/sendmail ;; stop) echo -n "Stopping sendmail: " killproc sendmail echo rm -f /var/lock/subsys/sendmail ;; *) echo "Usage: sendmail {start|stop}" exit 1 esac exit 0 Pop should install normally with no extra effort. It will just need the inetd entry for it with the virtuald part added. The inetd.conf entries for sendmail and pop: pop-3 stream tcp nowait root /usr/bin/virtuald virtuald /virtual/conf.pop in.qpop -s smtp stream tcp nowait root /usr/bin/virtuald virtuald /virtual/conf.mail sendmail -bs 10. Virtual other Any other service should be a similar procedure. ╖ Add the binary and the libraries to the virtual filesystem. ╖ Add it to /etc/inetd.conf. ╖ Create a /virtual/conf.service file. ╖ Create any virtual scripts that need to be made. I have experimented with both the samba package and have written a virtual poppassd through Eudora. Both work without any problems. If there is enough interest, I will add a section on installing virtual samba. 11. Conclusion Those are all the steps you need. I hope that this article meets with a positive response. Again mail any responses to Computer Resource Center. If you have a question or an update to the document let me know and I will add it. 12. FAQ Q1. Why are there no questions in this FAQ? A1. Because nobody has asked any yet.